Full range gesture system

ABSTRACT

This disclosure relates to an interactive display, having a front surface including a viewing area, and providing an input/output interface for a user of an electronic device. A planar light guide (PLG) disposed substantially parallel to the front surface, has a periphery at least coextensive with the viewing area. A light-emitting source (LES), disposed outside the periphery of the PLG, is optically coupled with a PLG input. The PLG outputs reflected light, in a direction substantially orthogonal to the front surface, by reflecting light received from the LES. A light collecting device (LCD) collects scattered light that results from interaction of the reflected light with a user-controlled object. The LCD redirects the collected scattered light toward one or more light sensors. A processor recognizes, from outputs of the light sensors, an instance of a user gesture, and controls the interactive display and/or electronic device, responsive to the user gesture.

TECHNICAL FIELD

This disclosure relates to techniques for gesture recognition, and, morespecifically, to an interactive display that provides a userinput/output interface, controlled responsively to a user's gesture,that may be made over a wide range of distances from the interactivedisplay.

DESCRIPTION OF THE RELATED TECHNOLOGY

Electromechanical systems (EMS) include devices having electrical andmechanical elements, actuators, transducers, sensors, optical components(such as mirrors and optical film layers) and electronics. EMS can bemanufactured at a variety of scales including, but not limited to,microscales and nanoscales. For example, microelectromechanical systems(MEMS) devices can include structures having sizes ranging from about amicron to hundreds of microns or more. Nanoelectromechanical systems(NEMS) devices can include structures having sizes smaller than a micronincluding, for example, sizes smaller than several hundred nanometers.Electromechanical elements may be created using deposition, etching,lithography, and/or other micromachining processes that etch away partsof substrates and/or deposited material layers, or that add layers toform electrical and electromechanical devices.

One type of electromechanical systems device is called aninterferometric modulator (IMOD). As used herein, the term IMOD orinterferometric light modulator refers to a device that selectivelyabsorbs and/or reflects light using the principles of opticalinterference. In some implementations, an IMOD may include a pair ofconductive plates, one or both of which may be transparent and/orreflective, wholly or in part, and capable of relative motion uponapplication of an appropriate electrical signal. In an implementation,one plate may include a stationary layer deposited on a substrate andthe other plate may include a reflective membrane separated from thestationary layer by an air gap. The position of one plate in relation toanother can change the optical interference of light incident on theIMOD. IMOD devices have a wide range of applications, and areanticipated to be used in improving existing products and creating newproducts, especially those with display capabilities, such as personalcomputers and personal electronic devices (PED's).

Increasingly, electronic devices such as personal computers and personalelectronic devices (PED's) provide for at least some user inputs to beprovided by means other than physical buttons, keyboards, and point andclick devices. For example, touch screen displays are increasinglyrelied upon for common user input functions. The display quality oftouch screen displays, however, can be degraded by contamination from auser's touch. Moreover, when the user's interaction with the device islimited to a small two dimensional space, as is commonly the case withtouch screen displays of, at least, PEDs, the user's input (touch) maybe required to be very precisely located in order to achieve a desiredresult. This results in slowing down or otherwise impairing the user'sability to interact with the device.

Accordingly, it is desirable to have a user interface that isresponsive, at least in part, to “gestures” by which is meant, theelectronic device senses and reacts in a deterministic way to grossmotions of a user's hand, digit, or hand-held object. The gestures maybe made proximate to, but, advantageously, not in direct physicalcontact with the electronic device.

Known gesture recognition systems include camera-based, ultrasound andprojective capacitive systems. Ultrasound displays suffer fromresolution issues; for example, circular motion is difficult to trackand individual fingers are difficult to identify. Projective capacitivesystems yield good resolution near and on the surface of a display butmay be resolution limited further than about a half inch from thedisplay surface. Camera-based systems may provide good resolution atlarge distances and adequate resolution to within an inch of the displaysurface. However, the cameras, placed on the periphery of the displaymay have a limited field of view. As a result, gesture recognition isdifficult to achieve at or near the display surface.

SUMMARY

The systems, methods and devices of the disclosure each have severalinnovative aspects, no single one of which is solely responsible for thedesirable attributes disclosed herein.

One innovative aspect of the subject matter described in this disclosureincludes an apparatus or electronic device that cooperates with aninteractive display to provide an input/output (I/O) interface to a userof the apparatus. The interactive display has a front surface thatincludes a viewing area. The electronic device may include theinteractive display or be electrically or wirelessly coupled to theinteractive display. The apparatus may include a processor, a planarlight guide, a light emitting source, a light collecting device, and aplurality of light sensors. The planar light guide may be disposedsubstantially parallel to the front surface and have a periphery atleast coextensive with the viewing area. The light-emitting source maybe disposed outside the periphery of the planar light guide and beoptically coupled with an input of the planar light guide. The planarlight guide may include a first light-turning arrangement that outputsreflected light, in a direction having a substantial componentorthogonal to the front surface, by reflecting light received from thelight-emitting source. The light collecting device may be configured tocollect scattered light, the collected scattered light resulting frominteraction of the reflected light with an object. The light collectingdevice may include a second light-turning arrangement that redirects thecollected scattered light toward one or more of the light sensors. Eachlight sensor may be configured to output, to the processor, a signalrepresentative of a characteristic of the redirected collected scatteredlight. The processor may be configured to recognize, from the output ofthe light sensors, an instance of a user gesture, and to control theinteractive display and/or the electronic device, responsive to the usergesture.

In an implementation the object may include one or more of a hand,finger, hand held object, and other object under control of the user.The collected scattered light may result from interaction of ambientlight with the object, the object being located at a position anywherein a range between about 0 and 500 mm from the front surface of theinteractive display. The light-emitting source may include alight-emitting diode. The emitted light may include infrared light.

In an implementation, the planar light guide and the light collectingdevice share a common light-turning arrangement. The planar light guideand the light collecting device may be a single, coplanar arrangement.

In another implementation, the planar light guide and the lightcollecting device may each be disposed in a separate plane.

In an implementation, the planar light collecting device may include alight guide. One or both of the first light-turning arrangement and thesecond light-turning arrangement may include a plurality of reflectivemicrostructures. One or both of the first light-turning arrangement andthe second light-turning arrangement may include one or more of amicrostructure for reflecting light, a holographic film or surfacerelief grating for turning light by diffraction and a surface roughnessthat turns light by scattering. The second light-turning arrangement mayreflect the collected scattered light toward one or more of the lightsensors.

In an implementation, the processor is configured to process image data,and the apparatus may further include a memory device that is configuredto communicate with the processor. The apparatus may include a drivercircuit configured to send at least one signal to the display, and acontroller configured to send at least a portion of the image data tothe driver circuit. The apparatus may further include an image sourcemodule configured to send the image data to the processor. The imagesource module may include one or more of a receiver, transceiver, andtransmitter. The apparatus may further include an input deviceconfigured to receive input data and to communicate the input data tothe processor.

In an implementation, a method includes recognizing, with a processor,an instance of a user gesture from a respective output of a plurality oflight sensors, and controlling, with the processor one or both of anelectronic device and an interactive display that provides aninput/output (I/O) interface for the electronic device, responsive tothe user gesture. The respective outputs of the plurality of lightsensors may result from: (i) emitting light into a planar light guidethat is disposed substantially parallel to a front surface of theinteractive, the planar light guide including a light turningarrangement; (ii) reflecting, with the light turning arrangement, theemitted light from the planar light guide in a direction having asubstantial component orthogonal to the front surface; (iii) collectingscattered light resulting from interaction of the reflected light withan object; (iv) redirecting collected scattered light toward theplurality of light sensors; and (v) outputting from each light sensor,to the processor, the respective output, representative of acharacteristic of the redirected collected scattered light.

In an implementation, an apparatus includes an interactive display,having a front surface including a viewing area, and providing aninput/output (I/O) interface for a user of an electronic device. Theapparatus may further include a processor; a planar light collectingdevice; and a plurality of light sensors disposed outside the peripheryof the planar light collecting device. The light collecting device maybe disposed substantially parallel to the front surface, may have aperiphery at least coextensive with the viewing area of the interactivedisplay and may be configured to collect incident light, the collectedincident light resulting from interaction of ambient light with anobject. The light collecting device may include a light-turningarrangement that redirects the collected incident light toward one ormore of the light sensors. Each light sensor may be configured tooutput, to the processor, a signal representative of a characteristic ofthe redirected collected incident light. The processor may be configuredto recognize, from the output of the light sensors, an instance of auser gesture, and to control one or both of the interactive display andthe electronic device, responsive to the user gesture.

The collected incident light may result from interaction of ambientlight with the object, the object being located at a position anywherein a range between about 0 and 500 mm from the front surface of theinteractive display.

Details of one or more implementations of the subject matter describedin this specification are set forth in the accompanying drawings and thedescription below. Although the examples provided in this summary areprimarily described in terms of MEMS-based displays, the conceptsprovided herein apply to other types of displays, such as organiclight-emitting diode (“OLED”) displays and field emission displays.Other features, aspects, and advantages will become apparent from thedescription, the drawings, and the claims. Note that the relativedimensions of the following figures may not be drawn to scale.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows an example of an isometric view depicting two adjacentpixels in a series of pixels of an interferometric modulator (IMOD)display device.

FIG. 2 shows an example of a system block diagram illustrating anelectronic device incorporating a 3×3 IMOD display.

FIG. 3 shows an example of a diagram illustrating movable reflectivelayer position versus applied voltage for the IMOD of FIG. 1.

FIG. 4 shows an example of a table illustrating various states of anIMOD when various common and segment voltages are applied.

FIG. 5A shows an example of a diagram illustrating a frame of displaydata in the 3×3 IMOD display of FIG. 2.

FIG. 5B shows an example of a timing diagram for common and segmentsignals that may be used to write the frame of display data illustratedin FIG. 5A.

FIG. 6A shows an example of a partial cross-section of the IMOD displayof FIG. 1.

FIGS. 6B-6E show examples of cross-sections of varying implementationsof IMODs.

FIG. 7 shows an example of a flow diagram illustrating a manufacturingprocess for an IMOD.

FIGS. 8A-8E show examples of cross-sectional schematic illustrations ofvarious stages in a method of making an IMOD.

FIG. 9A shows an example of a block diagram of an electronic devicehaving an interactive display according to an implementation.

FIGS. 9B-9D show an example of an arrangement including a planar lightguide, a light collecting device, a light emitting source, and lightsensors according to an implementation.

FIG. 9E shows an example of a planar light guide, according to animplementation, as viewed from the line E-E of FIG. 9C.

FIG. 9F shows an example of a light collecting device, according to animplementation, as viewed from the line F-F of FIG. 9C.

FIGS. 10A-10B show an example of an arrangement including a planardevice, a light emitting source, and light sensors according to animplementation.

FIG. 10C shows an example of the planar device as viewed from line C-Cof FIG. 10B.

FIG. 10D shows an example of the planar device as viewed from line D-Dof FIG. 10B.

FIGS. 11A-C show examples of microstructures for use in a light-turningarrangement according to some implementations.

FIGS. 12A-12B show examples of arrangements including a coplanar planarlight guide and light collecting device, light emitting sources, andlight sensors according to some implementations.

FIG. 13 shows an example of an arrangement including a planar device andlight sensors, according to an implementation.

FIG. 14 shows an example of a flow diagram illustrating a method forcontrolling an interactive display and/or an electronic device where theinteractive display provides an input/output (I/O) interface for theelectronic device.

FIGS. 15A and 15B show examples of system block diagrams illustrating adisplay device that includes a plurality of IMODs.

Like reference numbers and designations in the various drawings indicatelike elements.

DETAILED DESCRIPTION

The following description is directed to certain implementations for thepurposes of describing the innovative aspects of this disclosure.However, a person having ordinary skill in the art will readilyrecognize that the teachings herein can be applied in a multitude ofdifferent ways. The described implementations may be implemented in anydevice or system that can be configured to display an image, whether inmotion (e.g., video) or stationary (e.g., still image), and whethertextual, graphical or pictorial. More particularly, it is contemplatedthat the described implementations may be included in or associated witha variety of electronic devices such as, but not limited to: mobiletelephones, multimedia Internet enabled cellular telephones, mobiletelevision receivers, wireless devices, smartphones, Bluetooth® devices,personal data assistants (PDAs), wireless electronic mail receivers,hand-held or portable computers, netbooks, notebooks, smartbooks,tablets, printers, copiers, scanners, facsimile devices, GPSreceivers/navigators, cameras, MP3 players, camcorders, game consoles,wrist watches, clocks, calculators, television monitors, flat paneldisplays, electronic reading devices (i.e., e-readers), computermonitors, auto displays (including odometer and speedometer displays,etc.), cockpit controls and/or displays, camera view displays (such asthe display of a rear view camera in a vehicle), electronic photographs,electronic billboards or signs, projectors, architectural structures,microwaves, refrigerators, stereo systems, cassette recorders orplayers, DVD players, CD players, VCRs, radios, portable memory chips,washers, dryers, washer/dryers, parking meters, packaging (such as inelectromechanical systems (EMS), microelectromechanical systems (MEMS)and non-MEMS applications), aesthetic structures (e.g., display ofimages on a piece of jewelry) and a variety of EMS devices. Theteachings herein also can be used in non-display applications such as,but not limited to, electronic switching devices, radio frequencyfilters, sensors, accelerometers, gyroscopes, motion-sensing devices,magnetometers, inertial components for consumer electronics, parts ofconsumer electronics products, varactors, liquid crystal devices,electrophoretic devices, drive schemes, manufacturing processes andelectronic test equipment. Thus, the teachings are not intended to belimited to the implementations depicted solely in the Figures, butinstead have wide applicability as will be readily apparent to onehaving ordinary skill in the art.

Described herein below are new techniques for providing, on aninteractive display, a gesture-responsive user input/output (I/O)interface for an electronic device. “Gesture” as used herein broadlyrefers to a gross motion of a user's hand, digit, or hand-held object,or other object under control of the user. The motion may be madeproximate to, but not necessarily in direct physical contact with, theelectronic device. In some implementations, the electronic device sensesand reacts in a deterministic way to a user's gesture.

Particular implementations of the subject matter described in thisdisclosure can be implemented to realize one or more of the followingpotential advantages. The user's gesture may occur over a “full range”of view with respect to the interactive display. By “full range” ismeant that the gesture may be recognized, at a first extreme, even whenmade very close to, or in physical contact with, the interactivedisplay; in other words, “blind spots” exhibited by prior art camerasystems are avoided. At a second extreme, the gesture may be recognizedat a substantial distance, up to approximately 500 mm, from theinteractive display, which is not possible with known projectivecapacitive systems.

The above-described functionality is provided by a compact, low power,low cost solution which may include a light emitting source that emitslight toward a planar light guide. The planar light guide may bedisposed substantially parallel to a front surface of the interactivedisplay, and have a periphery at least coextensive with a viewing areaof the interactive display. The planar light guide may include a firstlight-turning arrangement that outputs reflected light, in a directionhaving a substantial component orthogonal to the front surface.

Advantageously, the light-turning arrangement operates by reflectingemitted light received from the light-emitting source that, in theabsence of the light-turning arrangement, would be totally internallyreflected by the planar light guide. Because the light-turningarrangement operates by reflection, the refractive index of thelight-turning arrangement is not critical, and turning of the emittedlight over a large range of angles is possible.

Although much of the description herein pertains to IMOD displays, manysuch implementations could be used to advantage in other types ofreflective displays, including but not limited to electrophoretic inkdisplays and displays based on electrowetting technology. Moreover,while the IMOD displays described herein generally include red, blue andgreen pixels, many implementations described herein could be used inreflective displays having other colors of pixels, such as havingviolet, yellow-orange and yellow-green pixels. Moreover, manyimplementations described herein could be used in reflective displayshaving more colors of pixels, such as having pixels corresponding to 4,5, or more colors. Some such implementations may include pixelscorresponding to red, blue, green and yellow. Alternativeimplementations may include pixels corresponding to at least red, blue,green, yellow and cyan.

An example of a suitable device, to which the described implementationsmay apply, is a reflective EMS or MEMS-based display device. Reflectivedisplay devices can incorporate IMODs to selectively absorb and/orreflect light incident thereon using principles of optical interference.IMODs can include an absorber, a reflector that is movable with respectto the absorber, and an optical resonant cavity defined between theabsorber and the reflector. The reflector can be moved to two or moredifferent positions, which can change the size of the optical resonantcavity and thereby affect the reflectance of the IMOD. The reflectancespectrums of IMODs can create fairly broad spectral bands which can beshifted across the visible wavelengths to generate different colors. Theposition of the spectral band can be adjusted by changing the thicknessof the optical resonant cavity. One way of changing the optical resonantcavity is by changing the position of the reflector.

FIG. 1 shows an example of an isometric view depicting two adjacentpixels in a series of pixels of an IMOD display device. The IMOD displaydevice includes one or more interferometric MEMS display elements. Inthese devices, the pixels of the MEMS display elements can be in eithera bright or dark state. In the bright (“relaxed,” “open” or “on”) state,the display element reflects a large portion of incident visible light,e.g., to a user. Conversely, in the dark (“actuated,” “closed” or “off”)state, the display element reflects little incident visible light. Insome implementations, the light reflectance properties of the on and offstates may be reversed. MEMS pixels can be configured to reflectpredominantly at particular wavelengths allowing for a color display inaddition to black and white.

The IMOD display device can include a row/column array of IMODs. EachIMOD can include a pair of reflective layers, i.e., a movable reflectivelayer and a fixed partially reflective layer, positioned at a variableand controllable distance from each other to form an air gap (alsoreferred to as an optical gap or cavity). The movable reflective layermay be moved between at least two positions. In a first position, i.e.,a relaxed position, the movable reflective layer can be positioned at arelatively large distance from the fixed partially reflective layer. Ina second position, i.e., an actuated position, the movable reflectivelayer can be positioned more closely to the partially reflective layer.Incident light that reflects from the two layers can interfereconstructively or destructively depending on the position of the movablereflective layer, producing either an overall reflective ornon-reflective state for each pixel. In some implementations, the IMODmay be in a reflective state when unactuated, reflecting light withinthe visible spectrum, and may be in a dark state when unactuated,absorbing and/or destructively interfering light within the visiblerange. In some other implementations, however, an IMOD may be in a darkstate when unactuated, and in a reflective state when actuated. In someimplementations, the introduction of an applied voltage can drive thepixels to change states. In some other implementations, an appliedcharge can drive the pixels to change states.

The depicted portion of the pixel array in FIG. 1 includes two adjacentIMODs 12. In the IMOD 12 on the left (as illustrated), a movablereflective layer 14 is illustrated in a relaxed position at apredetermined distance from an optical stack 16, which includes apartially reflective layer. The voltage V₀ applied across the IMOD 12 onthe left is insufficient to cause actuation of the movable reflectivelayer 14. In the IMOD 12 on the right, the movable reflective layer 14is illustrated in an actuated position near or adjacent the opticalstack 16. The voltage V_(bias) applied across the IMOD 12 on the rightis sufficient to maintain the movable reflective layer 14 in theactuated position.

In FIG. 1, the reflective properties of pixels 12 are generallyillustrated with arrows 13 indicating light incident upon the pixels 12,and light 15 reflecting from the pixel 12 on the left. Although notillustrated in detail, it will be understood by a person having ordinaryskill in the art that most of the light 13 incident upon the pixels 12will be transmitted through the transparent substrate 20, toward theoptical stack 16. A portion of the light incident upon the optical stack16 will be transmitted through the partially reflective layer of theoptical stack 16, and a portion will be reflected back through thetransparent substrate 20. The portion of light 13 that is transmittedthrough the optical stack 16 will be reflected at the movable reflectivelayer 14, back toward (and through) the transparent substrate 20.Interference (constructive or destructive) between the light reflectedfrom the partially reflective layer of the optical stack 16 and thelight reflected from the movable reflective layer 14 will determine thewavelength(s) of light 15 reflected from the pixel 12.

The optical stack 16 can include a single layer or several layers. Thelayer(s) can include one or more of an electrode layer, a partiallyreflective and partially transmissive layer and a transparent dielectriclayer. In some implementations, the optical stack 16 is electricallyconductive, partially transparent and partially reflective, and may befabricated, for example, by depositing one or more of the above layersonto a transparent substrate 20. The electrode layer can be formed froma variety of materials, such as various metals, for example indium tinoxide (ITO). The partially reflective layer can be formed from a varietyof materials that are partially reflective, such as various metals, suchas chromium (Cr), semiconductors, and dielectrics. The partiallyreflective layer can be formed of one or more layers of materials, andeach of the layers can be formed of a single material or a combinationof materials. In some implementations, the optical stack 16 can includea single semi-transparent thickness of metal or semiconductor whichserves as both an optical absorber and electrical conductor, whiledifferent, electrically more conductive layers or portions (e.g., of theoptical stack 16 or of other structures of the IMOD) can serve to bussignals between IMOD pixels. The optical stack 16 also can include oneor more insulating or dielectric layers covering one or more conductivelayers or an electrically conductive/optically absorptive layer.

In some implementations, the layer(s) of the optical stack 16 can bepatterned into parallel strips, and may form row electrodes in a displaydevice as described further below. As will be understood by one havingordinary skill in the art, the term “patterned” is used herein to referto masking as well as etching processes. In some implementations, ahighly conductive and reflective material, such as aluminum (Al), may beused for the movable reflective layer 14, and these strips may formcolumn electrodes in a display device. The movable reflective layer 14may be formed as a series of parallel strips of a deposited metal layeror layers (orthogonal to the row electrodes of the optical stack 16) toform columns deposited on top of posts 18 and an intervening sacrificialmaterial deposited between the posts 18. When the sacrificial materialis etched away, a defined gap 19, or optical cavity, can be formedbetween the movable reflective layer 14 and the optical stack 16. Insome implementations, the spacing between posts 18 may be approximately1-1000 um, while the gap 19 may be approximately less than 10,000Angstroms (Å).

In some implementations, each pixel of the IMOD, whether in the actuatedor relaxed state, is essentially a capacitor formed by the fixed andmoving reflective layers. When no voltage is applied, the movablereflective layer 14 remains in a mechanically relaxed state, asillustrated by the pixel 12 on the left in FIG. 1, with the gap 19between the movable reflective layer 14 and optical stack 16. However,when a potential difference, a voltage, is applied to at least one of aselected row and column, the capacitor formed at the intersection of therow and column electrodes at the corresponding pixel becomes charged,and electrostatic forces pull the electrodes together. If the appliedvoltage exceeds a threshold, the movable reflective layer 14 can deformand move near or against the optical stack 16. A dielectric layer (notshown) within the optical stack 16 may prevent shorting and control theseparation distance between the layers 14 and 16, as illustrated by theactuated pixel 12 on the right in FIG. 1. The behavior is the sameregardless of the polarity of the applied potential difference. Though aseries of pixels in an array may be referred to in some instances as“rows” or “columns,” a person having ordinary skill in the art willreadily understand that referring to one direction as a “row” andanother as a “column” is arbitrary. Restated, in some orientations, therows can be considered columns, and the columns considered to be rows.Furthermore, the display elements may be evenly arranged in orthogonalrows and columns (an “array”), or arranged in non-linear configurations,for example, having certain positional offsets with respect to oneanother (a “mosaic”). The terms “array” and “mosaic” may refer to eitherconfiguration. Thus, although the display is referred to as including an“array” or “mosaic,” the elements themselves need not be arrangedorthogonally to one another, or disposed in an even distribution, in anyinstance, but may include arrangements having asymmetric shapes andunevenly distributed elements.

FIG. 2 shows an example of a system block diagram illustrating anelectronic device incorporating a 3×3 IMOD display. The electronicdevice includes a processor 21 that may be configured to execute one ormore software modules. In addition to executing an operating system, theprocessor 21 may be configured to execute one or more softwareapplications, including a web browser, a telephone application, an emailprogram, or any other software application.

The processor 21 can be configured to communicate with an array driver22. The array driver 22 can include a row driver circuit 24 and a columndriver circuit 26 that provide signals to, for example, a display arrayor panel 30. The cross section of the IMOD display device illustrated inFIG. 1 is shown by the lines 1-1 in FIG. 2. Although FIG. 2 illustratesa 3×3 array of IMODs for the sake of clarity, the display array 30 maycontain a very large number of IMODs, and may have a different number ofIMODs in rows than in columns, and vice versa.

FIG. 3 shows an example of a diagram illustrating movable reflectivelayer position versus applied voltage for the IMOD of FIG. 1. For MEMSIMODs, the row/column (i.e., common/segment) write procedure may takeadvantage of a hysteresis property of these devices as illustrated inFIG. 3. An IMOD may use, in one example implementation, about a 10-voltpotential difference to cause the movable reflective layer, or mirror,to change from the relaxed state to the actuated state. When the voltageis reduced from that value, the movable reflective layer maintains itsstate as the voltage drops back below, in this example, 10 volts;however, the movable reflective layer does not relax completely untilthe voltage drops below 2 volts. Thus, a range of voltage, approximately3 to 7 volts, in this example, as shown in FIG. 3, exists where there isa window of applied voltage within which the device is stable in eitherthe relaxed or actuated state. This is referred to herein as the“hysteresis window” or “stability window.” For a display array 30 havingthe hysteresis characteristics of FIG. 3, the row/column write procedurecan be designed to address one or more rows at a time, such that duringthe addressing of a given row, pixels in the addressed row that are tobe actuated are exposed to a voltage difference of about, in thisexample, 10 volts, and pixels that are to be relaxed are exposed to avoltage difference of near zero volts. After addressing, the pixels canbe exposed to a steady state or bias voltage difference of approximately5 volts in this example, such that they remain in the previous strobingstate. In this example, after being addressed, each pixel sees apotential difference within the “stability window” of about 3-7 volts.This hysteresis property feature enables the pixel design, such as thatillustrated in FIG. 1, to remain stable in either an actuated or relaxedpre-existing state under the same applied voltage conditions. Since eachIMOD pixel, whether in the actuated or relaxed state, is essentially acapacitor formed by the fixed and moving reflective layers, this stablestate can be held at a steady voltage within the hysteresis windowwithout substantially consuming or losing power. Moreover, essentiallylittle or no current flows into the IMOD pixel if the applied voltagepotential remains substantially fixed.

In some implementations, a frame of an image may be created by applyingdata signals in the form of “segment” voltages along the set of columnelectrodes, in accordance with the desired change (if any) to the stateof the pixels in a given row. Each row of the array can be addressed inturn, such that the frame is written one row at a time. To write thedesired data to the pixels in a first row, segment voltagescorresponding to the desired state of the pixels in the first row can beapplied on the column electrodes, and a first row pulse in the form of aspecific “common” voltage or signal can be applied to the first rowelectrode. The set of segment voltages can then be changed to correspondto the desired change (if any) to the state of the pixels in the secondrow, and a second common voltage can be applied to the second rowelectrode. In some implementations, the pixels in the first row areunaffected by the change in the segment voltages applied along thecolumn electrodes, and remain in the state they were set to during thefirst common voltage row pulse. This process may be repeated for theentire series of rows, or alternatively, columns, in a sequentialfashion to produce the image frame. The frames can be refreshed and/orupdated with new image data by continually repeating this process atsome desired number of frames per second.

The combination of segment and common signals applied across each pixel(that is, the potential difference across each pixel) determines theresulting state of each pixel. FIG. 4 shows an example of a tableillustrating various states of an IMOD when various common and segmentvoltages are applied. As will be understood by one having ordinary skillin the art, the “segment” voltages can be applied to either the columnelectrodes or the row electrodes, and the “common” voltages can beapplied to the other of the column electrodes or the row electrodes.

As illustrated in FIG. 4 (as well as in the timing diagram shown in FIG.5B), when a release voltage VC_(REL) is applied along a common line, allIMOD elements along the common line will be placed in a relaxed state,alternatively referred to as a released or unactuated state, regardlessof the voltage applied along the segment lines, i.e., high segmentvoltage VS_(H) and low segment voltage VS_(L). In particular, when therelease voltage VC_(REL) is applied along a common line, the potentialvoltage across the modulator pixels (alternatively referred to as apixel voltage) is within the relaxation window (see FIG. 3, alsoreferred to as a release window) both when the high segment voltageVS_(H) and the low segment voltage VS_(L) are applied along thecorresponding segment line for that pixel.

When a hold voltage is applied on a common line, such as a high holdvoltage VC_(HOLD) _(_) _(H) or a low hold voltage VC_(HOLD) _(_) _(L),the state of the IMOD will remain constant. For example, a relaxed IMODwill remain in a relaxed position, and an actuated IMOD will remain inan actuated position. The hold voltages can be selected such that thepixel voltage will remain within a stability window both when the highsegment voltage VS_(H) and the low segment voltage VS_(L) are appliedalong the corresponding segment line. Thus, the segment voltage swing,i.e., the difference between the high VS_(H) and low segment voltageVS_(L), is less than the width of either the positive or the negativestability window.

When an addressing, or actuation, voltage is applied on a common line,such as a high addressing voltage VC_(ADD) _(_) _(H) or a low addressingvoltage VC_(ADD) _(_) _(L), data can be selectively written to themodulators along that line by application of segment voltages along therespective segment lines. The segment voltages may be selected such thatactuation is dependent upon the segment voltage applied. When anaddressing voltage is applied along a common line, application of onesegment voltage will result in a pixel voltage within a stabilitywindow, causing the pixel to remain unactuated. In contrast, applicationof the other segment voltage will result in a pixel voltage beyond thestability window, resulting in actuation of the pixel. The particularsegment voltage which causes actuation can vary depending upon whichaddressing voltage is used. In some implementations, when the highaddressing voltage VC_(ADD) _(_) _(H) is applied along the common line,application of the high segment voltage VS_(H) can cause a modulator toremain in its current position, while application of the low segmentvoltage VS_(L) can cause actuation of the modulator. As a corollary, theeffect of the segment voltages can be the opposite when a low addressingvoltage VC_(ADD) _(_) _(L) is applied, with high segment voltage VS_(H)causing actuation of the modulator, and low segment voltage VS_(L)having no effect (i.e., remaining stable) on the state of the modulator.

In some implementations, hold voltages, address voltages, and segmentvoltages may be used which produce the same polarity potentialdifference across the modulators. In some other implementations, signalscan be used which alternate the polarity of the potential difference ofthe modulators from time to time. Alternation of the polarity across themodulators (that is, alternation of the polarity of write procedures)may reduce or inhibit charge accumulation which could occur afterrepeated write operations of a single polarity.

FIG. 5A shows an example of a diagram illustrating a frame of displaydata in the 3×3 IMOD display of FIG. 2. FIG. 5B shows an example of atiming diagram for common and segment signals that may be used to writethe frame of display data illustrated in FIG. 5A. The signals can beapplied to a 3×3 array, similar to the array of FIG. 2, which willultimately result in the line time 60 e display arrangement illustratedin FIG. 5A. The actuated modulators in FIG. 5A are in a dark-state,i.e., where a substantial portion of the reflected light is outside ofthe visible spectrum so as to result in a dark appearance to, forexample, a viewer. Prior to writing the frame illustrated in FIG. 5A,the pixels can be in any state, but the write procedure illustrated inthe timing diagram of FIG. 5B presumes that each modulator has beenreleased and resides in an unactuated state before the first line time60 a.

During the first line time 60 a: a release voltage 70 is applied oncommon line 1; the voltage applied on common line 2 begins at a highhold voltage 72 and moves to a release voltage 70; and a low holdvoltage 76 is applied along common line 3. Thus, the modulators (common1, segment 1), (1,2) and (1,3) along common line 1 remain in a relaxed,or unactuated, state for the duration of the first line time 60 a, themodulators (2,1), (2,2) and (2,3) along common line 2 will move to arelaxed state, and the modulators (3,1), (3,2) and (3,3) along commonline 3 will remain in their previous state. With reference to FIG. 4,the segment voltages applied along segment lines 1, 2 and 3 will have noeffect on the state of the IMODs, as none of common lines 1, 2 or 3 arebeing exposed to voltage levels causing actuation during line time 60 a(i.e., VC_(REL)-relax and VC_(HOLD) _(_) _(L)-stable).

During the second line time 60 b, the voltage on common line 1 moves toa high hold voltage 72, and all modulators along common line 1 remain ina relaxed state regardless of the segment voltage applied because noaddressing, or actuation, voltage was applied on the common line 1. Themodulators along common line 2 remain in a relaxed state due to theapplication of the release voltage 70, and the modulators (3,1), (3,2)and (3,3) along common line 3 will relax when the voltage along commonline 3 moves to a release voltage 70.

During the third line time 60 c, common line 1 is addressed by applyinga high address voltage 74 on common line 1. Because a low segmentvoltage 64 is applied along segment lines 1 and 2 during the applicationof this address voltage, the pixel voltage across modulators (1,1) and(1,2) is greater than the high end of the positive stability window(i.e., the voltage differential exceeded a predefined threshold) of themodulators, and the modulators (1,1) and (1,2) are actuated. Conversely,because a high segment voltage 62 is applied along segment line 3, thepixel voltage across modulator (1,3) is less than that of modulators(1,1) and (1,2), and remains within the positive stability window of themodulator; modulator (1,3) thus remains relaxed. Also during line time60 c, the voltage along common line 2 decreases to a low hold voltage76, and the voltage along common line 3 remains at a release voltage 70,leaving the modulators along common lines 2 and 3 in a relaxed position.

During the fourth line time 60 d, the voltage on common line 1 returnsto a high hold voltage 72, leaving the modulators along common line 1 intheir respective addressed states. The voltage on common line 2 isdecreased to a low address voltage 78. Because a high segment voltage 62is applied along segment line 2, the pixel voltage across modulator(2,2) is below the lower end of the negative stability window of themodulator, causing the modulator (2,2) to actuate. Conversely, because alow segment voltage 64 is applied along segment lines 1 and 3, themodulators (2,1) and (2,3) remain in a relaxed position. The voltage oncommon line 3 increases to a high hold voltage 72, leaving themodulators along common line 3 in a relaxed state.

Finally, during the fifth line time 60 e, the voltage on common line 1remains at high hold voltage 72, and the voltage on common line 2remains at a low hold voltage 76, leaving the modulators along commonlines 1 and 2 in their respective addressed states. The voltage oncommon line 3 increases to a high address voltage 74 to address themodulators along common line 3. As a low segment voltage 64 is appliedon segment lines 2 and 3, the modulators (3,2) and (3,3) actuate, whilethe high segment voltage 62 applied along segment line 1 causesmodulator (3,1) to remain in a relaxed position. Thus, at the end of thefifth line time 60 e, the 3×3 pixel array is in the state shown in FIG.5A, and will remain in that state as long as the hold voltages areapplied along the common lines, regardless of variations in the segmentvoltage which may occur when modulators along other common lines (notshown) are being addressed.

In the timing diagram of FIG. 5B, a given write procedure (i.e., linetimes 60 a-60 e) can include the use of either high hold and addressvoltages, or low hold and address voltages. Once the write procedure hasbeen completed for a given common line (and the common voltage is set tothe hold voltage having the same polarity as the actuation voltage), thepixel voltage remains within a given stability window, and does not passthrough the relaxation window until a release voltage is applied on thatcommon line. Furthermore, as each modulator is released as part of thewrite procedure prior to addressing the modulator, the actuation time ofa modulator, rather than the release time, may determine the line time.Specifically, in implementations in which the release time of amodulator is greater than the actuation time, the release voltage may beapplied for longer than a single line time, as depicted in FIG. 5B. Insome other implementations, voltages applied along common lines orsegment lines may vary to account for variations in the actuation andrelease voltages of different modulators, such as modulators ofdifferent colors.

The details of the structure of IMODs that operate in accordance withthe principles set forth above may vary widely. For example, FIGS. 6B-6Eshow examples of cross-sections of varying implementations of IMODs,including the movable reflective layer 14 and its supporting structures.FIG. 6A shows an example of a partial cross-section of the IMOD displayof FIG. 1, where a strip of metal material, i.e., the movable reflectivelayer 14 is deposited on supports 18 extending orthogonally from thesubstrate 20. In FIG. 6B, the movable reflective layer 14 of each IMODis generally square or rectangular in shape and attached to supports ator near the corners, on tethers 32. In FIG. 6C, the movable reflectivelayer 14 is generally square or rectangular in shape and suspended froma deformable layer 34, which may include a flexible metal. Thedeformable layer 34 can connect, directly or indirectly, to thesubstrate 20 around the perimeter of the movable reflective layer 14.These connections are herein referred to as support posts. Theimplementation shown in FIG. 6C has additional benefits deriving fromthe decoupling of the optical functions of the movable reflective layer14 from its mechanical functions, which are carried out by thedeformable layer 34. This decoupling allows the structural design andmaterials used for the reflective layer 14 and those used for thedeformable layer 34 to be optimized independently of one another.

FIG. 6D shows another example of an IMOD, where the movable reflectivelayer 14 includes a reflective sub-layer 14 a. The movable reflectivelayer 14 rests on a support structure, such as support posts 18. Thesupport posts 18 provide separation of the movable reflective layer 14from the lower stationary electrode (i.e., part of the optical stack 16in the illustrated IMOD) so that a gap 19 is formed between the movablereflective layer 14 and the optical stack 16, for example when themovable reflective layer 14 is in a relaxed position. The movablereflective layer 14 also can include a conductive layer 14 c, which maybe configured to serve as an electrode, and a support layer 14 b. Inthis example, the conductive layer 14 c is disposed on one side of thesupport layer 14 b, distal from the substrate 20, and the reflectivesub-layer 14 a is disposed on the other side of the support layer 14 b,proximal to the substrate 20. In some implementations, the reflectivesub-layer 14 a can be conductive and can be disposed between the supportlayer 14 b and the optical stack 16. The support layer 14 b can includeone or more layers of a dielectric material, for example, siliconoxynitride (SiON) or silicon dioxide (SiO₂). In some implementations,the support layer 14 b can be a stack of layers, such as, for example, aSiO₂/SiON/SiO₂ tri-layer stack. Either or both of the reflectivesub-layer 14 a and the conductive layer 14 c can include, for example,an aluminum (Al) alloy with about 0.5% copper (Cu), or anotherreflective metallic material. Employing conductive layers 14 a, 14 cabove and below the dielectric support layer 14 b can balance stressesand provide enhanced conduction. In some implementations, the reflectivesub-layer 14 a and the conductive layer 14 c can be formed of differentmaterials for a variety of design purposes, such as achieving specificstress profiles within the movable reflective layer 14.

As illustrated in FIG. 6D, some implementations also can include a blackmask structure 23. The black mask structure 23 can be formed inoptically inactive regions (such as between pixels or under posts 18) toabsorb ambient or stray light. The black mask structure 23 also canimprove the optical properties of a display device by inhibiting lightfrom being reflected from or transmitted through inactive portions ofthe display, thereby increasing the contrast ratio. Additionally, theblack mask structure 23 can be conductive and be configured to functionas an electrical bussing layer. In some implementations, the rowelectrodes can be connected to the black mask structure 23 to reduce theresistance of the connected row electrode. The black mask structure 23can be formed using a variety of methods, including deposition andpatterning techniques. The black mask structure 23 can include one ormore layers. For example, in some implementations, the black maskstructure 23 includes a molybdenum-chromium (MoCr) layer that serves asan optical absorber, a layer, and an aluminum alloy that serves as areflector and a bussing layer, with a thickness in the range of about30-80 Å, 500-1000 Å, and 500-6000 Å, respectively. The one or morelayers can be patterned using a variety of techniques, includingphotolithography and dry etching, including, for example, carbontetrafluoromethane (CF₄) and/or oxygen (O₂) for the MoCr and SiO₂ layersand chlorine (Cl₂) and/or boron trichloride (BCl₃) for the aluminumalloy layer. In some implementations, the black mask 23 can be an etalonor interferometric stack structure. In such interferometric stack blackmask structures 23, the conductive absorbers can be used to transmit orbus signals between lower, stationary electrodes in the optical stack 16of each row or column. In some implementations, a spacer layer 35 canserve to generally electrically isolate the absorber layer 16 a from theconductive layers in the black mask 23.

FIG. 6E shows another example of an IMOD, where the movable reflectivelayer 14 is self-supporting. In contrast with FIG. 6D, theimplementation of FIG. 6E does not include support posts 18. Instead,the movable reflective layer 14 contacts the underlying optical stack 16at multiple locations, and the curvature of the movable reflective layer14 provides sufficient support that the movable reflective layer 14returns to the unactuated position of FIG. 6E when the voltage acrossthe IMOD is insufficient to cause actuation. The optical stack 16, whichmay contain a plurality of several different layers, is shown here forclarity including an optical absorber 16 a, and a dielectric 16 b. Insome implementations, the optical absorber 16 a may serve both as afixed electrode and as a partially reflective layer. In someimplementations, the optical absorber 16 a is an order of magnitude (tentimes or more) thinner than the movable reflective layer 14. In someimplementations, optical absorber 16 a is thinner than reflectivesub-layer 14 a.

In implementations such as those shown in FIGS. 6A-6E, the IMODsfunction as direct-view devices, in which images are viewed from thefront side of the transparent substrate 20, i.e., the side opposite tothat upon which the modulator is arranged. In these implementations, theback portions of the device (that is, any portion of the display devicebehind the movable reflective layer 14, including, for example, thedeformable layer 34 illustrated in FIG. 6C) can be configured andoperated upon without impacting or negatively affecting the imagequality of the display device, because the reflective layer 14 opticallyshields those portions of the device. For example, in someimplementations a bus structure (not illustrated) can be included behindthe movable reflective layer 14 which provides the ability to separatethe optical properties of the modulator from the electromechanicalproperties of the modulator, such as voltage addressing and themovements that result from such addressing. Additionally, theimplementations of FIGS. 6A-6E can simplify processing, such aspatterning.

FIG. 7 shows an example of a flow diagram illustrating a manufacturingprocess 80 for an IMOD, and FIGS. 8A-8E show examples of cross-sectionalschematic illustrations of corresponding stages of such a manufacturingprocess 80. In some implementations, the manufacturing process 80 can beimplemented to manufacture an electromechanical systems device such asIMODs of the general type illustrated in FIGS. 1 and 6. The manufactureof an electromechanical systems device also can include other blocks notshown in FIG. 7. With reference to FIGS. 1, 6 and 7, the process 80begins at block 82 with the formation of the optical stack 16 over thesubstrate 20. FIG. 8A illustrates such an optical stack 16 formed overthe substrate 20. The substrate 20 may be a transparent substrate suchas glass or plastic, it may be flexible or relatively stiff andunbending, and may have been subjected to prior preparation processes,such as cleaning, to facilitate efficient formation of the optical stack16. As discussed above, the optical stack 16 can be electricallyconductive, partially transparent and partially reflective and may befabricated, for example, by depositing one or more layers having thedesired properties onto the transparent substrate 20. In FIG. 8A, theoptical stack 16 includes a multilayer structure having sub-layers 16 aand 16 b, although more or fewer sub-layers may be included in someother implementations. In some implementations, one of the sub-layers 16a and 16 b can be configured with both optically absorptive andelectrically conductive properties, such as the combinedconductor/absorber sub-layer 16 a. Additionally, one or more of thesub-layers 16 a, 16 b can be patterned into parallel strips, and mayform row electrodes in a display device. Such patterning can beperformed by a masking and etching process or another suitable processknown in the art. In some implementations, one of the sub-layers 16 a,16 b can be an insulating or dielectric layer, such as sub-layer 16 bthat is deposited over one or more metal layers (e.g., one or morereflective and/or conductive layers). In addition, the optical stack 16can be patterned into individual and parallel strips that form the rowsof the display. It is noted that FIG. 8A8E may not be drawn to scale.For example, in some implementations, one of the sub-layers of theoptical stack, the optically absorptive layer, may be very thin,although sub-layers 16 a, 16 b are shown somewhat thick in FIGS. 8A-8E.

The process 80 continues at block 84 with the formation of a sacrificiallayer 25 over the optical stack 16. The sacrificial layer 25 is laterremoved (see block 90) to form the cavity 19 and thus the sacrificiallayer 25 is not shown in the resulting IMODs 12 illustrated in FIG. 1.FIG. 8B illustrates a partially fabricated device including asacrificial layer 25 formed over the optical stack 16. The formation ofthe sacrificial layer 25 over the optical stack 16 may includedeposition of a xenon difluoride (XeF₂)-etchable material such asmolybdenum (Mo) or amorphous silicon (a-Si), in a thickness selected toprovide, after subsequent removal, a gap or cavity 19 (see also FIGS. 1and 8E) having a desired design size. Deposition of the sacrificialmaterial may be carried out using deposition techniques such as physicalvapor deposition (PVD, which includes many different techniques, such assputtering), plasma-enhanced chemical vapor deposition (PECVD), thermalchemical vapor deposition (thermal CVD), or spin-coating.

The process 80 continues at block 86 with the formation of a supportstructure such as post 18, illustrated in FIGS. 1, 6 and 8C. Theformation of the post 18 may include patterning the sacrificial layer 25to form a support structure aperture, then depositing a material (suchas a polymer or an inorganic material such as silicon oxide) into theaperture to form the post 18, using a deposition method such as PVD,PECVD, thermal CVD, or spin-coating. In some implementations, thesupport structure aperture formed in the sacrificial layer can extendthrough both the sacrificial layer 25 and the optical stack 16 to theunderlying substrate 20, so that the lower end of the post 18 contactsthe substrate 20 as illustrated in FIG. 6A. Alternatively, as depictedin FIG. 8C, the aperture formed in the sacrificial layer 25 can extendthrough the sacrificial layer 25, but not through the optical stack 16.For example, FIG. 8E illustrates the lower ends of the support posts 18in contact with an upper surface of the optical stack 16. The post 18,or other support structures, may be formed by depositing a layer ofsupport structure material over the sacrificial layer 25 and patterningportions of the support structure material located away from aperturesin the sacrificial layer 25. The support structures may be locatedwithin the apertures, as illustrated in FIG. 8C, but also can, at leastpartially, extend over a portion of the sacrificial layer 25. As notedabove, the patterning of the sacrificial layer 25 and/or the supportposts 18 can be performed by a patterning and etching process, but alsomay be performed by alternative etching methods.

The process 80 continues at block 88 with the formation of a movablereflective layer or membrane such as the movable reflective layer 14illustrated in FIGS. 1, 6 and 8D. The movable reflective layer 14 may beformed by employing one or more deposition steps including, for example,reflective layer (such as aluminum, aluminum alloy, or other reflectivelayer) deposition, along with one or more patterning, masking, and/oretching steps. The movable reflective layer 14 can be electricallyconductive, and referred to as an electrically conductive layer. In someimplementations, the movable reflective layer 14 may include a pluralityof sub-layers 14 a, 14 b, 14 c as shown in FIG. 8D. In someimplementations, one or more of the sub-layers, such as sub-layers 14 a,14 c, may include highly reflective sub-layers selected for theiroptical properties, and another sub-layer 14 b may include a mechanicalsub-layer selected for its mechanical properties. Since the sacrificiallayer 25 is still present in the partially fabricated IMOD formed atblock 88, the movable reflective layer 14 is typically not movable atthis stage. A partially fabricated IMOD that contains a sacrificiallayer 25 also may be referred to herein as an “unreleased” IMOD. Asdescribed above in connection with FIG. 1, the movable reflective layer14 can be patterned into individual and parallel strips that form thecolumns of the display.

The process 80 continues at block 90 with the formation of a cavity,such as cavity 19 illustrated in FIGS. 1, 6 and 8E. The cavity 19 may beformed by exposing the sacrificial material 25 (deposited at block 84)to an etchant. For example, an etchable sacrificial material such as Moor amorphous Si may be removed by dry chemical etching, by exposing thesacrificial layer 25 to a gaseous or vaporous etchant, such as vaporsderived from solid XeF₂, for a period of time that is effective toremove the desired amount of material. The sacrificial material istypically selectively removed relative to the structures surrounding thecavity 19. Other etching methods, such as wet etching and/or plasmaetching, also may be used. Since the sacrificial layer 25 is removedduring block 90, the movable reflective layer 14 is typically movableafter this stage. After removal of the sacrificial material 25, theresulting fully or partially fabricated IMOD may be referred to hereinas a “released” IMOD.

FIG. 9A shows an example of a block diagram of an electronic devicehaving an interactive display according to an implementation. Apparatus900, which may be, for example a personal electronic device (PED), mayinclude interactive display 902 and processor 904. Interactive display902 may be a touch screen display, but this is not necessarily so.Processor 904 may be configured to control an output of interactivedisplay 902, responsive, at least in part, to user inputs. At least someof the user inputs may be made by way of gestures, which include grossmotions of a user's appendage, such as a hand or a finger, or a handheldobject or the like. The gestures may be located, with respect tointeractive display 902, at a wide range of distances. For example, agesture may be made proximate to, or even in direct physical contactwith interactive display 902. Alternatively, the gesture may be made ata substantial distance, up to, approximately, 500 mm from interactivedisplay 902.

Arrangement 930 (examples of which are described and illustrated hereinbelow) may be disposed over and substantially parallel to a frontsurface of interactive display 902. In an implementation, arrangement930 may be substantially transparent. Arrangement 930 may output one ormore signals responsive to a user gesture. Signals outputted byarrangement 930, via signal path 911, may be analyzed by processor 904to recognize an instance of a user gesture. Processor 904 may thencontrol display 902 responsive to the user gesture, by way of signalssent to interactive display 902 via signal path 913.

FIGS. 9B-9D show an example of an arrangement including a planar lightguide, a light collecting device, a light emitting source, and lightsensors according to an implementation. In the illustratedimplementation, arrangement 930 includes planar light guide 935, lightcollecting device 965, light emitting source 931, and light sensors 933.Referring now to FIG. 9B, which may be referred to as an elevation view,arrangement 930 is illustrated as being disposed above and substantiallyparallel to the front surface of display 902. In the illustratedimplementation, perimeters of planar light guide 935 and lightcollecting device 965 are substantially coextensive with a perimeter ofdisplay 902. Advantageously, the perimeters of planar light guide 935and light collecting device 965 are coextensive with, or are larger thanand fully envelop, the perimeter of interactive display 902.

In the illustrated implementation, two light sensors 933 are provided;however, three or more light sensors may be provided in otherimplementations. Light sensors 933 may include photosensitive elements,such as photodiodes, phototransistors, charge coupled device (CCD)arrays, complementary metal oxide semiconductor (CMOS) arrays or othersuitable devices operable to output a signal representative of acharacteristic of detected visible, infrared (IR) and/or ultraviolet(UV) light. Light sensors 933 may output signals representative of oneor more characteristics of detected light. For example, thecharacteristics may include intensity, directionality, frequency,amplitude, amplitude modulation, and/or other properties.

In some implementations, light sensors 933 may be optically coupled tolight collecting device 965. In the illustrated implementation, lightsensors 933 are disposed at the periphery of light collecting device965. However, alternative configurations are within the contemplation ofthe present disclosure. For example, light sensors 933 may be remotefrom light collecting device 965, in which case light detected by lightsensors 933 may be transmitted from light collecting device 965 byadditional optical elements such as, for example, one or more opticalfibers

In an implementation, light-emitting source 931 may be a light-emittingdiode (LED) configured to emit primarily infrared light. However, anytype of light source may be used. For example, light emitting source 931may include one or more organic light emitting devices (“OLEDs”), lasers(for example, diode lasers or other laser sources), hot or cold cathodefluorescent lamps, incandescent or halogen light sources. In theillustrated implementation, light emitting source 931 is disposed at theperiphery of planar light guide 935. However, alternative configurationsare within the contemplation of the present disclosure. For example,light emitting source 931 may be remote from planar light guide 935 andlight produced by light emitting source 931 may be transmitted to planarlight guide 935 by additional optical elements such as, for example, oneor more optical fibers, reflectors, etc. In some implementations, lightemitting source 931 may be configured to emit light over a solid angle.The solid angle may be selected to enhance gesture recognitionreliability, for example. In the illustrated implementation, onelight-emitting source 931 is provided; however, two or morelight-emitting sources may be provided in other implementations.

Referring now to FIGS. 9C and 9D, which may be referred to,respectively, as a plan view and a perspective view, light-emittingsource 931 and light sensors 933 are shown disposed proximate to andoutside of the periphery of planar light guide 935. Light emittingsource 931 may be optically coupled with an input of planar light guide935 and may be configured to emit light toward planar light guide 935 ina direction having a substantial component parallel to the front surfaceof interactive display 902.

FIG. 9E shows an example of a planar light guide, according to animplementation, as viewed from line E-E of FIG. 9C. For clarity ofillustration, interactive display 902 and internal details of lightcollecting device 965 are omitted from FIG. 9E. Planar light guide 935may include a substantially transparent, relatively thin, overlaydisposed on, or above and proximate to, the front surface of interactivedisplay 902. In one implementation, for example, planar light guide 935may be approximately 0.5 mm thick, while having a planar area in anapproximate range of tens or hundreds of square centimeters. Planarlight guide 935 may include a thin plate composed of a transparentmaterial such as glass or plastic, having a front surface 937 and a rearsurface 939, which may be substantially flat, parallel surfaces.

The transparent material may have an index of refraction greater than 1.For example, the index of refraction may be in the range of about 1.4 to1.6. The index of refraction of the transparent material determines acritical angle ‘α’ with respect to a normal of front surface 937 suchthat a light ray intersecting front surface 937 at an angle less than‘α’ will pass through front surface 937 but a light ray having anincident angle with respect to front surface 937 greater than ‘α’ willundergo total internal reflection (TIR).

In the illustrated implementation, planar light guide 935 includes alight-turning arrangement that reflects emitted light received fromlight emitting source 931 in a direction having a substantial componentorthogonal to front surface 937. More particularly, at least asubstantial fraction of reflected light 942 intersects front surface 937at an angle to the normal less than critical angle ‘α’. As a result,such reflected light 942 does not undergo TIR, but instead may betransmitted through front surface 937.

In an implementation, planar light guide 935 may have a light-turningarrangement that includes a number of reflective microstructures 936that redirect emitted light 941, received from light-emitting source 931in a direction having a substantial component orthogonal to the frontsurface of interactive display 902. In the absence of microstructures936, substantially all of the light received from light emitting source931 would undergo TIR, following a path, for example, illustrated bydashed line 945. As a result of operation of reflective microstructures936 however, emitted light 941 received from light emitting source 931that would otherwise undergo TIR, is instead outputted from planar lightguide 935 as reflected light 942 in a direction having a substantialcomponent orthogonal to the front surface of interactive display 902.

FIG. 9F shows an example of a light collecting device according to animplementation as viewed from the line F-F of FIG. 9C. For clarity ofillustration, interactive display 902 and internal details of planarlight guide 935 are omitted from FIG. 9F. Light collecting device 965may include a substantially transparent, relatively thin, overlaydisposed on, or above and proximate to, front surface 937 of planarlight guide 935. In some implementations, light collecting device 965may be made of the same material as planar light guide 935, but this isnot necessarily so. In an implementation, planar light guide 935 may beseparated from light collecting device 965 by a decoupling layer (notillustrated) that may include, for example a pressure sensitiveadhesive. The decoupling layer may have refractive index that is lowerthan both the refractive index of planar light guide 935 and therefractive index of light collecting device 965. In one implementation,for example, light collecting device 965 may be approximately 0.5 mmthick, while having a planar area similar to that of planar light guide935. Light collecting device 965 may include a thin plate composed of atransparent material such as glass or plastic, having a front surface967 and a rear surface 969, which may be substantially flat, parallelsurfaces.

The transparent material may have an index of refraction greater than 1.For example, the index of refraction may be in the range of about 1.4 to1.6. The index of refraction of the transparent material determines acritical angle ‘α’ such that a light ray intersecting front surface 967at an angle to the surface normal less than ‘α’ will pass through frontsurface 967 but a light ray having an incident angle with respect tofront surface 937 greater than ‘α’ will undergo TIR.

As illustrated in FIG. 9F, when object 950 interacts with reflectedlight 942, scattered light 944, resulting from the interaction, may bedirected toward light collecting device 965. Object 950 may be, forexample, a user's appendage, such as a hand or a finger, or it may beany physical object, hand-held or otherwise under control of the user,but is herein referred to, for simplicity, as the “object.” Lightcollecting device 965 may, as illustrated, be a planar light guideconfigured similarly to planar light guide 935, and include alight-turning arrangement that includes a number of reflectivemicrostructures 966.

Light collecting device 965 may be configured to collect scattered light944. Advantageously, light collecting device 965 includes alight-turning arrangement that redirects the scattered light 944,collected by light collecting device 965 toward one or more of lightsensors 933. The redirected collected scattered light 946 may be turnedin a direction having a substantial component parallel to the frontsurface of interactive display 902. More particularly, at least asubstantial fraction of redirected collected scattered light 946intersects front surface 967 and back surface 969 only at an angle tonormal greater than critical angle ‘α’ and, therefore, undergoes TIR. Asa result, such redirected collected scattered light 946 does not passthrough front surface 967 or back surface 969 and, instead, reaches oneor more of light sensors 933. Each light sensor 933 may be configured todetect one or more characteristics of the redirected collected scatteredlight 946, and output, to processor 204, a signal representative of thedetected characteristics. For example, the characteristics may includeintensity, directionality, frequency, amplitude, amplitude modulation,and/or other properties.

Referring again to FIG. 9A, processor 904 may be configured to receive,from light sensors 933, signals representative of the detectedcharacteristics, via signal path 911. Processor 904 may make acomparison of signals received from all light sensors 933 to calculate aposition of object 950, in, at least, a two dimensional plane parallelto the front surface of interactive display 902. In one implementation,for example, light sensors 933 may be disposed at opposing positions onthe periphery of light collecting device 965. Each light sensor 933 maygenerate output signals in response to received light that, at least inthe aggregate, enable the position, size and/or shape of object 950 tobe determined. For example, by comparing the output signal of eachrespective light sensor 933, processor 904 may determine the position ofobject 950, in a plane generally parallel with the front surface ofinteractive display 902. As a further example, a motion of the objectmay cause light received by the light sensors to produce a signalpattern. The processor may be configured to analyze the signal patternand determine when the signal pattern is indicative of a characteristicof a particular user gesture. The signal pattern may includecharacteristics such as signal intensity and/or waveform. For example,the intensity of a signal generated at all detectors may change as theobject comes closer to the screen. As a further example, a pulse-likewaveform may be detected by one or more detectors when, for example, ahand with fingers spaced apart moves across a plane generally parallelwith the front surface of interactive display 902.

Processor 904 may be configured to recognize, from the output signals oflight sensors 933, an instance of a user gesture. Moreover, processor904 may control one or both of interactive display 902 and/or otherelements of the electronic device 900, responsive to the user gesture.For example, an image displayed on interactive display 902 may be causedto be scrolled up or down, rotated, enlarged, or otherwise modified. Inaddition, the processor 904 may be configured to control other aspectsof electronic device 900, responsive to the user gesture, such as, forexample, changing a volume setting, turning power off, placing orterminating a call, launching or terminating a software application,etc.

In the above described implementation, two separate light-turningarrangements are provided, a first light-turning arrangement beingincluded in planar light guide 935, and a second light-turningarrangement being included in light collecting device 965. In such animplementation, planar light guide 935 and light emitting source 931 maybe disposed in a first plane, whereas light collecting device 965 may bedisposed in a second, different, plane. Illustrating an example of adifferent implementation, FIGS. 10A-10B show an example of anarrangement including a planar device, a light emitting source, andlight sensors according to an implementation. In the illustratedimplementation, a single planar device 1035 provides similarfunctionality as planar light guide 935 and light collecting device 965.

Planar device 1035 may be disposed in a single plane, coplanar withlight emitting source 931 and light sensors 933. Referring now to FIGS.10A and 10B, which may be referred to, respectively, as an elevationview and a perspective view, planar device 1035 may be disposed aboveand substantially parallel to the front surface of interactive display902. In the illustrated implementation, a perimeter of planar device1035 is substantially coextensive with a perimeter of display 902.Advantageously, the perimeter of arrangement 1035 is coextensive with,or is larger than and fully envelops the perimeter of interactivedisplay 902. Light-emitting source 931 and light sensors 933 may bedisposed proximate to and outside of the periphery of planar device1035. Light emitting source 931 may be optically coupled with an inputof planar device 1035 and may be configured to emit light toward planardevice 1035 in a direction having a substantial component parallel tothe front surface of interactive display 902. Light sensors 933 may beoptically coupled with an output of planar device 1035 and may beconfigured to detect light output from planar device 1035 in a directionhaving a substantial component parallel to the front surface ofinteractive display 902.

FIG. 10C shows an example of the planar device as viewed from line C-Cof FIG. 10B. For clarity of illustration, interactive display 902 isomitted from FIG. 10C. Planar device 1035 may include a substantiallytransparent, relatively thin, overlay disposed on, or above andproximate to, the front surface of interactive display 902. In oneimplementation, for example, planar device 1035 may be approximately 0.5mm thick, while having a planar area in an approximate range of tens orhundreds of square centimeters. Planar light guide 1035 may include athin plate composed of a transparent material such as glass or plastic,having a front surface 1037 and a rear surface 1039, which may besubstantially flat, parallel surfaces.

The transparent material may have an index of refraction greater than 1.For example, the index of refraction may be in the range of about 1.4 to1.6. The index of refraction of the transparent material determines acritical angle ‘α’ with respect to a normal of front surface 1037 suchthat a light ray intersecting front surface 1037 at an angle less than‘α’ will pass through front surface 1037 but a light ray having anincident angle with respect to front surface 1037 greater than ‘α’ willundergo total internal reflection (TIR).

In the illustrated implementation, planar device 1035 includes alight-turning arrangement that reflects emitted light received fromlight emitting source 931 in a direction having a substantial componentorthogonal to front surface 1037. More particularly, at least asubstantial fraction of reflected light 942 intersects front surface1037 at an angle to the normal less than critical angle ‘α’. As aresult, such reflected light 942 does not undergo TIR, but instead maybe transmitted through front surface 1037. It will be appreciated thatreflected light 942 may be transmitted through front surface 1037 at awide variety of angles. As a result, some of reflected light 942 may bedirected away from object 950, toward, or away from, a user's field ofvision, for example.

In an implementation, planar device 1035 may have a light-turningarrangement that includes a number of reflective microstructures 1036.The microstructures 1036 can all be identical, or have different shapes,sizes, structures, etc., in various implementations. Some examples ofthe microstructures 1036 are discussed in details below with referenceto FIGS. 11A-C. Microstructures 1036 may redirect emitted light 941 suchthat at least a substantial fraction of reflected light 942 intersectsfront surface 1037 at an angle to normal less than critical angle ‘α.

FIG. 10D shows an example of the planar device as viewed from line D-Dof FIG. 10B. For clarity of illustration, interactive display 902 isomitted from FIG. 10D. As illustrated in FIG. 10D, when object 950interacts with reflected light 942, scattered light 944, resulting fromthe interaction, may be directed toward planar device 1035. Planardevice 1035 may, as illustrated, include a light-turning arrangementthat includes a number of reflective microstructures 1066. Reflectivemicrostructures 1066 may be configured similarly to reflectivemicrostructures 1036, or be the same physical elements, but this is notnecessarily so.

As illustrated in FIG. 10D, when object 950 interacts with reflectedlight 942, scattered light 944, resulting from the interaction, may bedirected toward planar device 1035. Planar device 1035 may be configuredto collect scattered light 944. Advantageously, light collecting device1035 includes a light-turning arrangement that redirects the scatteredlight 944, collected by light collecting device 1035 toward one or moreof light sensors 933. The redirected collected scattered light 946 maybe turned in a direction having a substantial component parallel to thefront surface of interactive display 902. More particularly, at least asubstantial fraction of redirected collected scattered light 946intersects front surface 1037 and back surface 1039 only at an angle tonormal greater than critical angle ‘α’ and, therefore, undergoes TIR. Asa result, such redirected collected scattered light 946 does not passthrough front surface 1037 or back surface 1039 and, instead, reachesone or more of light sensors 933. Each light sensor 933 may beconfigured to detect one or more characteristics of the redirectedcollected scattered light 946, and output, to processor 204, a signalrepresentative of the detected characteristics. For example, thecharacteristics may include intensity, directionality, frequency,amplitude, amplitude modulation, and/or other properties.

FIG. 11A-C shows examples of microstructures for use in a light-turningarrangement according to some implementations. FIGS. 11A, 11B, and 11Cshow, respectively, an elevation view, a plan view, and a perspectiveview of microstructures 1101, 1103 and 1105. It will be appreciated thatthe illustration shows a highly magnified view of the microstructures,which will ordinarily be quite small, for example, in someimplementations having a height of approximately 1 to 10 μm and a width3 to 50 μm. Each microstructure, advantageously, will have one or morereflective surfaces, for example reflective surfaces 1102, 1104, and1106 that are configured to redirect light. More particularly, incidentlight directed in a direction having a substantial component parallel tothe front surface of interactive display 902 may be reflected in adirection having a substantial component orthogonal to the front surfaceof interactive display 902. Similarly, incident light directed in adirection having a substantial component orthogonal to the front surfaceof interactive display 902 may be reflected in a direction having asubstantial component parallel to the front surface of interactivedisplay in. It will be appreciated that many geometries of suchmicrostructures are possible, and the examples provided in FIGS. 11A-11Care merely illustrative of a few possible implementations.

In some implementations microstructures such as those illustrated inFIGS. 11A-11C may be formed by printing successive layers and structureson top of each other in sheets. In other implementations, embossingand/or molding techniques may be used to create the microstructures. Insome implementations, a reflective surface may be selectively providedby metallizing a glass substrate, for example. Reflective surfaces 1102,1104, and 1106 may be prepared using photolithography and wet chemicaletching techniques, for example. In some implementations, reflectivesurfaces 1102, 1104, and 1106 may be fabricated into a SiON layerdeposited on a glass substrate. In such implementations, masks may beused, and a thin metal layer (about 500-1000 Angstroms thick, forexample) maybe deposited only on the reflective surfaces.

Instead of, or in addition to, microstructures, such as thoseillustrated in FIG. 11, other turning arrangements are within thecontemplation of the present disclosure, including, for example,holographic film and surface relief grating that turn light bydiffraction or surface roughness that turns light by scattering.

In the above described implementations of arrangement 930, a particularexample of an arrangement of a single light emitting source 931 and twolight sensors 933 was illustrated. However, many other possibleconfigurations are within the contemplation of the present disclosure.For example, FIGS. 12A-12B show examples of arrangements including lightemitting sources and light sensors according to some implementations. Inone implementation, referring now to FIG. 12A, a single light emittingsource 931 and three light sensors 933 are each disposed at a respectivecorner of a rectangular arrangement 1230A. In another implementation,referring now to FIG. 12B, each of four single light emitting sources931 is disposed at a respective corner of a rectangular arrangement 930,while each of four light sensors 933 is disposed along a respective sideof the rectangular arrangement 1230B. It will be appreciated that theexamples provided in FIG. 12 are merely illustrative of a fewconfigurations, and that other configurations are possible. For example,one or more light sensors 933 and light emitting sources 931 may becollocated. Moreover, although the arrangements illustrated arerectangular, this is not necessarily so.

In the above described implementations of arrangement 930, referring nowto FIG. 9F, redirected collected scattered light 946 light that isdetected by light sensors 933 originates from light emitting source 931.The present inventors have appreciated that a gesture recognition systemconsistent with the present disclosure does not necessarily requirelight emitting source 931. For example, FIG. 13 shows an example of anarrangement including a planar device and light sensors, according to animplementation. In the illustrated implementation, planar arrangement1330 is operable to output one or more signals responsive to a usergesture even in the absence of light emitting source 931 (or if lightemitting source 931 is not illuminated). Planar arrangement 1330 may besubstantially similar to the above-described arrangement 930, exceptthat light emitting source 931 is omitted or not illuminated. In theillustrated implementation, interaction of object 950 and light fromambient light source 1370 produces shadows. Ambient light source 1370may be the sun, for example, or an artificial ambient light source.Advantageously, planar arrangement 1330 includes a light-turningarrangement that redirects incident light, including shadowed light,represented by lines 1344, toward one or more of light sensors 933. Eachlight sensor 933 may be configured to detect one or more characteristicsof the redirected shadowed light 1346, and output, to processor 904, asignal representative of the detected characteristics. As describedhereinabove, processor 904 may make a comparison of signals receivedfrom all light sensors 933 to calculate a position of object 950, in, atleast, a two dimensional plane parallel planar arrangement 1330.

FIG. 14 shows an example of a flow diagram illustrating a method forcontrolling an interactive display and/or an electronic device where theinteractive display provides an input/output (I/O) interface for theelectronic device. At block 1410, light from a light emitting source maybe emitted. The light emitting source may include, for example, an LED,emitting light at a visible, IR or UV wavelength and be opticallycoupled to an input aperture of a planar light guide. The interactivedisplay may have a front surface including a viewing area. The planarlight guide may be disposed substantially parallel to the front surfaceand have a periphery at least coextensive with the viewing area of theinteractive display, and include a light-turning arrangement

At block 1420, the emitted light may be reflected with the light-turningarrangement so as to output reflected light, in a direction having asubstantial component orthogonal to the front surface. The firstlight-turning arrangement may include a number of reflectivemicrostructures, as described herein above.

At block 1430, scattered light may be collected, the collected scatteredlight resulting from interaction of the reflected light with an object.The scattered light may be collected by a light collecting device thatincludes a second light-turning arrangement that redirects the collectedscattered light toward one or more light sensors. The secondlight-turning arrangement may include a number of reflectivemicrostructures, as described herein above.

At block 1440, at least one signal representative of a characteristic ofthe redirected collected scattered light may be outputted from eachlight sensor. For example, light sensors may output, to a processor,signals representative of one or more characteristics of detected light,such as intensity, directionality, frequency, amplitude, amplitudemodulation, and/or other properties.

At block 1450, an instance of a user gesture may be recognized from thelight sensor signals. For example, a motion of the object may causelight received by the light sensors to produce a signal pattern. Theprocessor may be configured to analyze the signal pattern and determinewhen the signal pattern is indicative of a characteristic of aparticular user gesture.

At block 1460 an interactive display and/or the electronic device may becontrolled, responsive to the user gesture. For example, the processormay be configured to cause an image displayed on the interactive displayto be scrolled up or down, rotated, enlarged, or otherwise modified.Alternatively, or in addition, the processor may be configured tocontrol other aspects of the electronic device, responsive to the usergesture. For example, the processor may be configured to change a volumesetting, power off the electronic device, place or terminating a call,launch or terminate a software application, etc., responsive to the usergesture.

FIGS. 15A and 15B show examples of system block diagrams illustrating adisplay device 40 that includes arrangement 930 for gesture recognition.The display device 40 can be, for example, a smart phone, a cellular ormobile telephone. However, the same components of the display device 40or slight variations thereof are also illustrative of various types ofdisplay devices such as televisions, tablets, e-readers, hand-helddevices and portable media players.

The display device 40 includes a housing 41, a display 30, arrangement930, an antenna 43, a speaker 45, an input device 48 and a microphone46. The housing 41 can be formed from any of a variety of manufacturingprocesses, including injection molding, and vacuum forming. In addition,the housing 41 may be made from any of a variety of materials,including, but not limited to: plastic, metal, glass, rubber andceramic, or a combination thereof. The housing 41 can include removableportions (not shown) that may be interchanged with other removableportions of different color, or containing different logos, pictures, orsymbols.

The display 30 may be any of a variety of displays, including abi-stable or analog display, as described herein. The display 30 alsocan be configured to include a flat-panel display, such as plasma, EL,OLED, STN LCD, or TFT LCD, or a non-flat-panel display, such as a CRT orother tube device. In addition, the display 30 can include an IMODdisplay, as described herein. The arrangement 930 may be an arrangementsubstantially as described herein.

The components of the display device 40 are schematically illustrated inFIG. 15B. The display device 40 includes a housing 41 and can includeadditional components at least partially enclosed therein. For example,the display device 40 includes a network interface 27 that includes anantenna 43 which is coupled to a transceiver 47. The transceiver 47 isconnected to a processor 21, which is connected to conditioning hardware52. The conditioning hardware 52 may be configured to condition a signal(e.g., filter a signal). The conditioning hardware 52 is connected to aspeaker 45 and a microphone 46. The processor 21 is also connected to aninput device 48 and a driver controller 29. The driver controller 29 iscoupled to a frame buffer 28, and to an array driver 22, which in turnis coupled to a display array 30. In some implementations, a powersupply 50 can provide power to substantially all components in theparticular display device 40 design.

In this example, the display device 40 also includes processor 904,which may be configured for communication with arrangement 930 via, forexample, routing wires, and may be configured for controlling the touchsensor device 900. In the illustrated implementation, processor 904 isshown separately from, for example processor 21 and drive controller 29.It will be appreciated, however, that the functionality of processor904, as discussed herein above, may be incorporated into processor 21and/or drive controller 29, or, as further example, into a hostprocessor (not shown). Processor 904 may be configured to recognize,from signals received from arrangement 930, an instance of a usergesture. Processor 904 may then control display array 30 responsive tothe user gesture. The network interface 27 includes the antenna 43 andthe transceiver 47 so that the display device 40 can communicate withone or more devices over a network. The network interface 27 also mayhave some processing capabilities to relieve, for example, dataprocessing requirements of the processor 21. The antenna 43 can transmitand receive signals. In some implementations, the antenna 43 transmitsand receives RF signals according to the IEEE 16.11 standard, includingIEEE 16.11(a), (b), or (g), or the IEEE 802.11 standard, including IEEE802.11a, b, g, n, and further implementations thereof. In some otherimplementations, the antenna 43 transmits and receives RF signalsaccording to the BLUETOOTH standard. In the case of a cellulartelephone, the antenna 43 is designed to receive code division multipleaccess (CDMA), frequency division multiple access (FDMA), time divisionmultiple access (TDMA), Global System for Mobile communications (GSM),GSM/General Packet Radio Service (GPRS), Enhanced Data GSM Environment(EDGE), Terrestrial Trunked Radio (TETRA), Wideband-CDMA (W-CDMA),Evolution Data Optimized (EV-DO), 1×EV-DO, EV-DO Rev A, EV-DO Rev B,High Speed Packet Access (HSPA), High Speed Downlink Packet Access(HSDPA), High Speed Uplink Packet Access (HSUPA), Evolved High SpeedPacket Access (HSPA+), Long Term Evolution (LTE), AMPS, or other knownsignals that are used to communicate within a wireless network, such asa system utilizing 3G or 4G technology. The transceiver 47 canpre-process the signals received from the antenna 43 so that they may bereceived by and further manipulated by the processor 21. The transceiver47 also can process signals received from the processor 21 so that theymay be transmitted from the display device 40 via the antenna 43.

In some implementations, the transceiver 47 can be replaced by areceiver. In addition, in some implementations, the network interface 27can be replaced by an image source, which can store or generate imagedata to be sent to the processor 21. The processor 21 can control theoverall operation of the display device 40. The processor 21 receivesdata, such as compressed image data from the network interface 27 or animage source, and processes the data into raw image data or into aformat that is readily processed into raw image data. The processor 21can send the processed data to the driver controller 29 or to the framebuffer 28 for storage. Raw data typically refers to the information thatidentifies the image characteristics at each location within an image.For example, such image characteristics can include color, saturationand gray-scale level.

The processor 21 can include a microcontroller, CPU, or logic unit tocontrol operation of the display device 40. The conditioning hardware 52may include amplifiers and filters for transmitting signals to thespeaker 45, and for receiving signals from the microphone 46. Theconditioning hardware 52 may be discrete components within the displaydevice 40, or may be incorporated within the processor 21 or othercomponents.

The driver controller 29 can take the raw image data generated by theprocessor 21 either directly from the processor 21 or from the framebuffer 28 and can re-format the raw image data appropriately for highspeed transmission to the array driver 22. In some implementations, thedriver controller 29 can re-format the raw image data into a data flowhaving a raster-like format, such that it has a time order suitable forscanning across the display array 30. Then the driver controller 29sends the formatted information to the array driver 22. Although adriver controller 29, such as an LCD controller, is often associatedwith the system processor 21 as a stand-alone Integrated Circuit (IC),such controllers may be implemented in many ways. For example,controllers may be embedded in the processor 21 as hardware, embedded inthe processor 21 as software, or fully integrated in hardware with thearray driver 22.

The array driver 22 can receive the formatted information from thedriver controller 29 and can re-format the video data into a parallelset of waveforms that are applied many times per second to the hundreds,and sometimes thousands (or more), of leads coming from the display'sx-y matrix of pixels.

In some implementations, the driver controller 29, the array driver 22,and the display array 30 are appropriate for any of the types ofdisplays described herein. For example, the driver controller 29 can bea conventional display controller or a bi-stable display controller(such as an IMOD controller). Additionally, the array driver 22 can be aconventional driver or a bi-stable display driver (such as an IMODdisplay driver). Moreover, the display array 30 can be a conventionaldisplay array or a bi-stable display array (such as a display includingan array of IMODs). In some implementations, the driver controller 29can be integrated with the array driver 22. Such an implementation canbe useful in highly integrated systems, for example, mobile phones,portable-electronic devices, watches or small-area displays.

In some implementations, the input device 48 can be configured to allow,for example, a user to control the operation of the display device 40.The input device 48 can include a keypad, such as a QWERTY keyboard or atelephone keypad, a button, a switch, a rocker, a touch-sensitivescreen, or a pressure- or heat-sensitive membrane. The microphone 46 canbe configured as an input device for the display device 40. In someimplementations, voice commands through the microphone 46 can be usedfor controlling operations of the display device 40.

The power supply 50 can include a variety of energy storage devices. Forexample, the power supply 50 can be a rechargeable battery, such as anickel-cadmium battery or a lithium-ion battery. In implementationsusing a rechargeable battery, the rechargeable battery may be chargeableusing power coming from, for example, a wall socket or a photovoltaicdevice or array. Alternatively, the rechargeable battery can bewirelessly chargeable. The power supply 50 also can be a renewableenergy source, a capacitor, or a solar cell, including a plastic solarcell or solar-cell paint. The power supply 50 also can be configured toreceive power from a wall outlet.

In some implementations, control programmability resides in the drivercontroller 29 which can be located in several places in the electronicdisplay system. In some other implementations, control programmabilityresides in the array driver 22. The above-described optimization may beimplemented in any number of hardware and/or software components and invarious configurations.

The various illustrative logics, logical blocks, modules, circuits andalgorithm steps described in connection with the implementationsdisclosed herein may be implemented as electronic hardware, computersoftware, or combinations of both. The interchangeability of hardwareand software has been described generally, in terms of functionality,and illustrated in the various illustrative components, blocks, modules,circuits and steps described above. Whether such functionality isimplemented in hardware or software depends upon the particularapplication and design constraints imposed on the overall system.

The hardware and data processing apparatus used to implement the variousillustrative logics, logical blocks, modules and circuits described inconnection with the aspects disclosed herein may be implemented orperformed with a general purpose single- or multi-chip processor, adigital signal processor (DSP), an application specific integratedcircuit (ASIC), a field programmable gate array (FPGA) or otherprogrammable logic device, discrete gate or transistor logic, discretehardware components, or any combination thereof designed to perform thefunctions described herein. A general purpose processor may be amicroprocessor, or, any conventional processor, controller,microcontroller, or state machine. A processor also may be implementedas a combination of computing devices, such as a combination of a DSPand a microprocessor, a plurality of microprocessors, one or moremicroprocessors in conjunction with a DSP core, or any other suchconfiguration. In some implementations, particular steps and methods maybe performed by circuitry that is specific to a given function.

In one or more aspects, the functions described may be implemented inhardware, digital electronic circuitry, computer software, firmware,including the structures disclosed in this specification and theirstructural equivalents thereof, or in any combination thereof.Implementations of the subject matter described in this specificationalso can be implemented as one or more computer programs, i.e., one ormore modules of computer program instructions, encoded on a computerstorage media for execution by, or to control the operation of, dataprocessing apparatus.

If implemented in software, the functions may be stored on ortransmitted over as one or more instructions or code on acomputer-readable medium. The steps of a method or algorithm disclosedherein may be implemented in a processor-executable software modulewhich may reside on a computer-readable medium. Computer-readable mediaincludes both computer storage media and communication media includingany medium that can be enabled to transfer a computer program from oneplace to another. A storage media may be any available media that may beaccessed by a computer. By way of example, and not limitation, suchcomputer-readable media may include RAM, ROM, EEPROM, CD-ROM or otheroptical disk storage, magnetic disk storage or other magnetic storagedevices, or any other medium that may be used to store desired programcode in the form of instructions or data structures and that may beaccessed by a computer. Also, any connection can be properly termed acomputer-readable medium. Disk and disc, as used herein, includescompact disc (CD), laser disc, optical disc, digital versatile disc(DVD), floppy disk, and blu-ray disc where disks usually reproduce datamagnetically, while discs reproduce data optically with lasers.Combinations of the above also may be included within the scope ofcomputer-readable media. Additionally, the operations of a method oralgorithm may reside as one or any combination or set of codes andinstructions on a machine readable medium and computer-readable medium,which may be incorporated into a computer program product.

Various modifications to the implementations described in thisdisclosure may be readily apparent to those skilled in the art, and thegeneric principles defined herein may be applied to otherimplementations without departing from the spirit or scope of thisdisclosure. Thus, the claims are not intended to be limited to theimplementations shown herein, but are to be accorded the widest scopeconsistent with this disclosure, the principles and the novel featuresdisclosed herein. The word “exemplary” is used exclusively herein tomean “serving as an example, instance, or illustration.” Anyimplementation described herein as “exemplary” is not necessarily to beconstrued as preferred or advantageous over other possibilities orimplementations. Additionally, a person having ordinary skill in the artwill readily appreciate, the terms “upper” and “lower” are sometimesused for ease of describing the figures, and indicate relative positionscorresponding to the orientation of the figure on a properly orientedpage, and may not reflect the proper orientation of an IMOD asimplemented.

Certain features that are described in this specification in the contextof separate implementations also can be implemented in combination in asingle implementation. Conversely, various features that are describedin the context of a single implementation also can be implemented inmultiple implementations separately or in any suitable subcombination.Moreover, although features may be described above as acting in certaincombinations and even initially claimed as such, one or more featuresfrom a claimed combination can in some cases be excised from thecombination, and the claimed combination may be directed to asubcombination or variation of a subcombination.

Similarly, while operations are depicted in the drawings in a particularorder, a person having ordinary skill in the art will readily recognizethat such operations need not be performed in the particular order shownor in sequential order, or that all illustrated operations be performed,to achieve desirable results. Further, the drawings may schematicallydepict one more example processes in the form of a flow diagram.However, other operations that are not depicted can be incorporated inthe example processes that are schematically illustrated. For example,one or more additional operations can be performed before, after,simultaneously, or between any of the illustrated operations. In certaincircumstances, multitasking and parallel processing may be advantageous.Moreover, the separation of various system components in theimplementations described above should not be understood as requiringsuch separation in all implementations, and it should be understood thatthe described program components and systems can generally be integratedtogether in a single software product or packaged into multiple softwareproducts. Additionally, other implementations are within the scope ofthe following claims. In some cases, the actions recited in the claimscan be performed in a different order and still achieve desirableresults.

What is claimed is:
 1. An apparatus comprising: an interactive display,having a first front surface including a viewing area, and providing aninput/output (I/O) interface for a user of an electronic device; aprocessor; a planar light guide disposed substantially parallel to thefirst front surface and having a periphery at least coextensive with theviewing area, the planar light guide being disposed proximate to and infront of the first front surface, and having a rear surface proximate tothe first front surface and a second front surface opposite to the rearsurface; a planar light collecting device disposed in front of the firstfront surface; a light-emitting source disposed outside the periphery ofthe planar light guide, the light-emitting source being opticallycoupled with an input of the planar light guide and not opticallycoupled with the planar light collecting device; and a plurality oflight sensors disposed outside the periphery of the planar light guide,the plurality of light sensors being optically coupled with an output ofthe planar light collecting device and not optically coupled with theplanar light guide; wherein, light from an image displayed on theinteractive display passes through the first front surface, the rearsurface and the second front surface; the planar light guide includes afirst light-turning arrangement that outputs reflected light through thesecond front surface, by reflecting emitted light received from thelight-emitting source; the light collecting device is configured tocollect scattered light, the collected scattered light resulting frominteraction of the reflected light with an object, irrespective ofwhether the object is located at a substantial distance, up toapproximately 500 mm, from the interactive display, located proximate tothe interactive display or in physical contact with the interactivedisplay; the light collecting device includes a second light-turningarrangement that redirects the collected scattered light toward one ormore of the light sensors; each light sensor is configured to output, tothe processor, a signal representative of a characteristic of theredirected collected scattered light; and the processor is configured torecognize, from the output of the light sensors, an instance of a usergesture, and to control one or both of the interactive display and theelectronic device, responsive to the user gesture, the control of theinteractive display including one or more of causing an image displayedon the interactive display to be scrolled up or down, rotated, enlarged,or otherwise modified, and the control of the electronic deviceincluding one or more of changing a volume setting, placing orterminating a call, launching or terminating a software application. 2.The apparatus of claim 1, wherein the object includes one or more of ahand, finger, hand held object, and other object under control of theuser.
 3. The apparatus of claim 1, wherein the light-emitting sourceincludes a light-emitting diode.
 4. The apparatus of claim 1, whereinthe emitted light includes infrared light.
 5. The apparatus of claim 1,wherein the planar light guide and the light collecting device are eachdisposed in a separate plane.
 6. The apparatus of claim 1, wherein theplanar light collecting device includes a light guide.
 7. The apparatusof claim 1, wherein one or both of the first light-turning arrangementand the second light-turning arrangement includes a plurality ofreflective microstructures.
 8. The apparatus of claim 1, wherein one orboth of the first light-turning arrangement and the second light-turningarrangement includes one or more of a microstructure for reflectinglight, a holographic film or surface relief grating for turning light bydiffraction and a surface roughness that turns light by scattering. 9.The apparatus of claim 1, wherein the second light-turning arrangementreflects the collected scattered light toward one or more of the lightsensors.
 10. The apparatus of claim 1, wherein the processor isconfigured to process image data, and the apparatus further includes amemory device that is configured to communicate with the processor. 11.The apparatus of claim 10, further comprising: a driver circuitconfigured to send at least one signal to the display; and a controllerconfigured to send at least a portion of the image data to the drivercircuit.
 12. The apparatus of claim 10, further including an imagesource module configured to send the image data to the processor,wherein the image source module includes one or more of a receiver,transceiver, and transmitter.
 13. The apparatus of claim 10, furthercomprising: an input device configured to receive input data and tocommunicate the input data to the processor.
 14. A method comprising:recognizing, with a processor, an instance of a user gesture from arespective output of a plurality of light sensors, and; controlling,with the processor one or both of an electronic device and aninteractive display that provides an input/output (I/O) interface forthe electronic device, responsive to the user gesture, wherein, therespective outputs of the plurality of light sensors result fromemitting light from a light emitting source into a planar light guidethat is disposed substantially parallel to a first front surface of theinteractive display, proximate to and in front of the first frontsurface, the planar light guide including a light turning arrangementand having a rear surface proximate to the first front surface and asecond front surface opposite to the rear surface such that light froman image displayed on the interactive display passes through the firstfront surface, the rear surface and the second front surface;reflecting, with the light turning arrangement, the emitted light fromthe planar light guide through the second front surface; collectingscattered light resulting from interaction of the reflected light withan object with a planar light collecting device disposed in front of thefirst front surface, irrespective of whether the object is located at asubstantial distance, up to approximately 500 mm, from the interactivedisplay, located proximate to the interactive display or in physicalcontact with the interactive display, the light emitting source beingoptically coupled with an input of the planar light guide and notoptically coupled with the planar light collecting device; redirectingcollected scattered light toward the plurality of light sensors, theplurality of light sensors being optically coupled with an output of theplanar light collecting device and not optically coupled with the planarlight guide; outputting from each light sensor, to the processor, therespective output, representative of a characteristic of the redirectedcollected scattered light; and the controlling of the interactivedisplay including one or more of causing an image displayed on theinteractive display to be scrolled up or down, rotated, enlarged, orotherwise modified, the controlling of the electronic device includingone or more of changing a volume setting placing or terminating a call,launching or terminating a software application.
 15. The method of claim14, wherein the object includes one or more of a hand, finger, hand heldobject, and other object under control of the user.
 16. The method ofclaim 14, wherein a light collecting device is configured to collect thelight, the planar light guide and the light collecting device sharing acommon light-turning arrangement.
 17. The method of claim 14, whereinthe planar light collecting device includes a light guide.
 18. Themethod of claim 14, wherein a second light-turning arrangement reflectsthe collected scattered light toward one or more of the light sensors.19. An apparatus comprising: an interactive display, having a firstfront surface including a viewing area, and providing an input/output(I/O) interface for a user of an electronic device; a planar light guidedisposed substantially parallel to the first front surface and having aperiphery at least coextensive with the viewing area, the planar lightguide being proximate to and in front of the first front surface, andhaving a rear surface proximate to the first front surface and a secondfront surface opposite to the rear surface; a planar light collectingdevice disposed in front of the first front surface; a light-emittingsource disposed outside the periphery of the planar light guide, thelight-emitting source being optically coupled with an input of theplanar light guide and not optically coupled with the planar lightcollecting device; a plurality of light sensors disposed outside theperiphery of the planar light guide, the plurality of light sensorsbeing optically coupled with an output of the planar light collectingdevice and not optically coupled with the planar light guide; and meansfor recognizing, from outputs of the light sensors, an instance of auser gesture, and for controlling one or both of the interactive displayand the electronic device, responsive to the user gesture; wherein,light from an image displayed on the interactive display passes throughthe first front surface, the rear surface and the second front surface;the planar light guide includes a first light-turning arrangement thatoutputs reflected light, through the second front surface, by reflectingemitted light received from the light-emitting source; the lightcollecting device is configured to collect scattered light, thecollected scattered light resulting from interaction of the reflectedlight with an object, irrespective of whether the object is located at asubstantial distance, up to approximately 500 mm, from the interactivedisplay, located proximate to the interactive display or in physicalcontact with the interactive display; the light collecting deviceincludes a second light-turning arrangement that redirects the collectedscattered light toward one or more of the light sensors; each lightsensor is configured to output a signal representative of acharacteristic of the redirected collected scattered light; and thecontrolling of the interactive display including one or more of causingan image displayed on the interactive display to be scrolled up or down,rotated, enlarged, or otherwise modified, and the controlling of theelectronic device including one or more of changing a volume setting,placing or terminating a call, launching or terminating a softwareapplication.
 20. The apparatus of claim 19, wherein the object includesone or more of a hand, finger, hand held object, and other object undercontrol of the user.
 21. The apparatus of claim 19, wherein one or bothof the first light-turning arrangement and the second light-turningarrangement includes a plurality of reflective microstructures.
 22. Anon-transitory computer-readable storage medium having stored thereoninstructions which, when executed by a processor, cause the processor toperform operations, the operations comprising: recognizing an instanceof a user gesture from a respective output of a plurality of lightsensors, and controlling one or both of an electronic device and aninteractive display that provides an input/output (I/O) interface anelectronic device, responsive to the user gesture, the interactivedisplay having a first front surface including a viewing area; whereinthe interactive display has a first front surface including a viewingarea, and provides an input/output (I/O) interface for the electronicdevice; the planar light guide is disposed substantially parallel to thefirst front surface, proximate to and in front of the first frontsurface, has a rear surface proximate to the first front surface and asecond front surface opposite to the rear surface such that light froman image displayed on the interactive display passes through the firstfront surface, the rear surface and the second front surface; the planarlight guide has a periphery at least coextensive with the viewing areaof the interactive display and a second front surface opposite to thefirst front surface, and includes a first light-turning arrangement thatoutputs reflected light, through the second front surface, by reflectingemitted light received from the light-emitting source; a planar lightcollecting device, disposed in front of the first front surface,collects scattered light, the collected scattered light resulting frominteraction of the reflected light with an object, the light collectingdevice including a second light-turning arrangement that redirects thecollected scattered light toward a plurality of light sensors,irrespective of whether the object is located at a substantial distance,up to approximately 500 mm, from the interactive display, locatedproximate to the interactive display or in physical contact with theinteractive display; a light emitting source emits light, the lightemitting source being optically coupled with an input of the planarlight guide and not optically coupled with the planar light collectingdevice; the plurality of sensors are optically coupled with an output ofthe planar light collecting device and not optically coupled with theplanar light guide; the respective outputs of the plurality of lightsensors result from each light sensor outputting, to the processor, asignal representative of a characteristic of the redirected collectedscattered light; and controlling of the interactive display includes oneor more of causing an image displayed on the interactive display to bescrolled up or down, rotated, enlarged, or otherwise modified, and thecontrolling of the electronic device including one or more of changing avolume setting, powering off the electronic device, placing orterminating a call, launching or terminating a software application. 23.The computer-readable storage medium of claim 22, wherein the objectincludes one or more of a hand, finger, hand held object, and otherobject under control of the user.
 24. The computer-readable storagemedium of claim 22, wherein the planar light collecting device includesa light guide.
 25. The computer-readable storage medium of claim 22,wherein the second light-turning arrangement reflects the collectedscattered light toward one or more of the plurality of light sensors.